As computers have increased in capabilities the demand for computing resources has also grown. In fact, the demand for computing resources has grown to the point that, in many cases, far outstrips the capabilities of a single computer to deliver resources to a user or application. For example, enterprises have been aware of this issue for many years and continue to buy racks of servers, storage arrays, or other computing resources at great cost to satisfy their needs for computing resources. In addition, many consumers employ external hard disk drives to store massive amounts of personal media data because their personal computers lack sufficient capacity. There are many reasons that drive the demand and proliferation of computing resources including legislation that affect enterprises, applications, ubiquitous digital cameras, media players, and countless other reasons. Industry has responded to the demand by producing products that provide computer resources to users and applications where the computing resource products are no longer centralized to a single computer. Furthermore, the industry is tending toward a distribution of computing resources where individual computer systems offer their capabilities or services to other users and applications where on example includes NAS file servers.
Yet another trend in the industry is to disaggregate resources into their constituent resource device elements, for example USB disk drives or SAN systems. However, these approaches do not address combining a number of resource elements together into a coherent virtual whole from the perspective of an arbitrary set of resource consumers, but rather these approaches still follow a centralized approach.
Even though the industry combines computing resources into a centralized set of capabilities or services as in SAN systems, it is still desirable to have the computing resources exist as individual resource nodes. Centralized resources imply further costs to due to the expense of the additional computer hardware and infrastructure; introduce yet another point of potential failure beyond the computing resources; create a bottleneck that all resource consumers must pass through, and so on. Disaggregated resources that comprise a collection of individual resource nodes that work independently but operate as a whole without a centralized controller or manager alleviate many of these problems. Individual resource nodes carry a smaller per unit price tag reducing incremental cost and offering stronger scalability, introduce no additional points of failure, do not require out-of-band communication increasing responsiveness, and operate in parallel increasing performance. Some known examples of distributed resources include clusters or applications like SETI@Home that offer CPU bandwidth as the computer resource. These examples are “distributed resources” where they rely on complete computer systems rather than individual computing resources and operate at an application level rather than at a resource device level.
Offering true disaggregated resources that comprise a number of resource nodes has a number of critical problems that must be overcome in order to deliver a solid disaggregated solution. First, networks are inherently unreliable and have latency; consequently resource nodes might loose connectivity with a resource consumer. Connectivity is important where the state or coherency of the resource from the perspective of a resource consumer is important. Second, multiple resource consumers can have different “views” of a disaggregated resource from each other; therefore, each resource consumer can have a different resource map used to access the disaggregated resource. Furthermore, most computer systems require access to a resource at a very fine level of granularity well below the resource device level. For example, when a CPU addresses memory, it attempts to reference a single byte or word. Such fine level granularity access is difficult in a disaggregated resource because a resource consumer does not necessarily have a sufficiently detailed and complete map of the disaggregated resource. Third, the organization of the resource nodes can be quite complex depending on a number of factors including type of resource involved, the roles or responsibilities of each resource node, resource node location, or other implementation specific information. Fourth, operating systems have to interpret the disaggregated resource as if it were locally connected in order to provide applications seamless, transparent integration with the computing environment.
A number of attempts have been made previously to provide a solid solution addressing the problems encountered when building a disaggregated resource. BitTorrent™, a peer-to-peer file transfer system, offers redundant file storage as a distributed resource where each resource node is complete computer system offering file storage. BitTorrent™ places redundant copies of data on multiple peers to alleviate some of the risk of an unreliable network; however, the peers are outside the control of the ultimate user so the user does not know if a peer is lost. Any owner of a BitTorrent™ system can take down their computer reducing the overall performance for a remote unknown user. Although each BitTorrent™ user has a different view of the network based on connectivity, a user can not, in a deterministic fashion, determine the extent and usability of the system. Unfortunately, BitTorrent™ is an application level protocol and does not provide a transparent solution of a storage resource that allows an operating system or application to read and write data at will. Cleary, BitTorrent™ offers some utility for high level file transfers; it is not suitable for consumer or enterprise system due to the lack of control, determinism, fine level access, or performance. BitTorrent™ and other peer-to-peer systems have not addressed the need for resource consumers to understand a complete map for a coherent disaggregated resource at an elemental level.
Hitachi's U.S. Pat. No. 4,890,227 offers a resource management system for operating systems of large scale computers. The memory, CPUs, I/O channels, and storage devices represent disaggregated resources. The management system relies on a set of policies that are continually updated and deleted to equitably and autonomously assign resources to process units (programs, threads, or tasks). The Hitachi patent does not address issues of resource coherency over an unreliable communication path or issues of multiple resource consumers (process units) having different views of the resources because a centralized management system handles all the resources for the process units. The Hitachi patent does not provide insight on how a resource consumer will manage and access resource nodes of a disaggregated resource that extend beyond the core computer. Even though the Hitachi patent addresses more elemental resources, the centralized resource management system does not allow resource consumers build their own view of the system or to function independently. Furthermore, the Hitachi patent offers no insight how to address fine level of structure of a resource.
Microsoft's U.S. Pat. No. 6,912,622 attempts to resolve some of problems associated with a distributed resource that are similar to the problems associated with a disaggregated resource where the distributed resource is a peer-to-peer system. The Microsoft patent uses an underlying statistical assumption regarding the probability of a first peer knowing a second peer's ID based on the “distance” between the first peer's ID and the second peer's ID. Through this structure, the Microsoft patent offers an efficient peer-to-peer name resolution system which allows a peer to keep track of and to find other peers by a useable organization scheme resulting in a map of the system. However, the structure only offers a way to access peers across a peer-to-peer network, but does not offer developers a way to access a fine level of detail within a peer as required by a truly disaggregated resource. Furthermore, the Microsoft patent does not address the need for a resource consumer to know the coherency of the disaggregated resource as resource nodes lose connectivity. In a peer-to-peer network, if a peer drops out, other peers don't necessary care. However, if the a resource node representing a CPU, memory, or storage device drops out of connectivity, all resource consumers using the disaggregated resource will need to know. Finally, the Microsoft patent does not address more elemental resource nodes.
Adaptec's U.S. Pat. No. 6,922,688 offers a method of accessing data objects where portions of the object are found through obtaining referential maps comprising logical storage locations and physical maps comprising the physical storage locations associated with the logical storage locations. Although the patent teaches how to access data objects distributed across a plurality of physical locations, it does not enable disaggregated resources, how to access such resources, how to operate disaggregated resources as one functional whole resource, or how to maintain a disaggregated resource over an unreliable network. In addition, although one aspect of the present invention comprises a split map, that map is a split map of disaggregated resources not a split map of data objects. Moreover, as discussed below, the term resource, as used herein, excludes data objects.
None of the previously presented examples fully address the problems encountered for building and accessing disaggregated resources. A more complete solution handles unreliable communications, resource consumer and resource node independence, resource coherency, fine level access to the resource, and applies to many types of elemental resources rather than create a solution for a single type of resource. Such a solution would have the following characteristics:                A disaggregated resource would comprise independent, efficient resource nodes that do not necessarily communicate with each other and do not require out-of-band communications        The resource nodes provide information about their role in the disaggregated resource to resource consumers that request the information        Resource consumers discover and access the resources without accessing extraneous systems        Resource consumers construct their own view of the disaggregated resource based on information from the resource nodes        A map of the resource provides access to a fine level of granularity to the resource at or below the resource device level        
Thus, there remains a considerable need apparatus for disaggregated resources and for methods of accessing disaggregated resources.